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ABSTRACT 

The report describes a model used to evaluate gifted 
programs. The model is said to rely on three sources of data: (1) 
structured narrative observation (the Gifted Class Activities 
Observation Scale); (2) teacher interviews; and (3) pupil interviews. 
(Sample forms appended.) The application of theory with this 
procedure resulted in an evaluation model featuring input 
(identification process, cognitive student growth, levels of 
thinking, classroom conditions, and attitude toward program) and such 
program data sources as student achievement, aptitude tast data, 
principal and supervisor recommendations, outside evaluation, and 
pupil questionnaires. It is concluded that the model and related 
instruments provide for a viable, yet not overly burdensome, 
evaluation. (CL) 
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Evaluation of programs for gifted students poses some unique problems 

and challenges for educators. Traditional evaluation models which rely almost 

solely on standardized test results are recognized as one of the least valid 

assessment techniques to use with a group of gifted learners. The problem 

of regression to the mean, which works so conveniently to the advantage of 

programs dealing with low achieving students, presents significant measurement 

problems when dealing with a population of students who can be expected to 

regress statistically downward. Additionally, conventionally normed tests 

tend to be unreliable at the extremes of their norms, and it is difficult to 

show much growth on standardized tests when scores are already at the upper 
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limits of the tests. ' 

Most of the standard books on teaching the gifted ignore the problems 
of evaluation altogether and focus instead on the myriad wayt, of developing 
programs for Lhese special learners. * These diverse programs traditionally 
suggested for gifted youngsters offer focus on higher level thought processes, 
completion of unique products, and attitudinal changes, all of v^hich are 
notoriously resistant to typical evaluation procedures. 



P a s s ow , ha r ry ed . The Gifted and The Talented: T; iei r Education and 
Development , 78th Yearbook National Society for The Study of Education, 1979. 
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Renzulli, Joseph. A Guidebook for Evaluating Programs for The Gifted 
& Talented . N/SLTI, 1975. 

^Barbe, Walter B. and Joseph Renzulli tul. Psychology and Education of 
The Gifted : New York: Lrvington Publishers, 1975. 

''Gallagher, James J. Teaching The Gifted Child , hoston: Allyn & Bacon, 
Inc., 1975. 
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Despite the problems that evaluation poses for gifted programs, the 
need lor such data for future planning is indisputable, it is with an eye 
toward program planning and future program development that the current 
evaluation model was conceived. In our model we have attempted to combine 
the need for objective data related to cognitive outcomes with the equally 
important data related to classroon atmosphere and attitudes. We have looked 
upon the evaluation model as an evolutional y one .which has changed already 
from Its original form, in the sections ol this report that follow, we will: 
1) describe the model as it was used to evaluate che 1979-80 program; 2) 
describe the model as it has been refined and modified to evaluate the 1980-81 
program; and 3) present the evaluative data from the 1979-80 program. 
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The 1979-80 Model for Evaluating Gifted Programs 

The evaluation model developed for 1979-80 program «.>us intentionally 
open-ended to allow the evaluators to absess the most useful types of data 
to collect. Three basic sources of data were used: 1) the structured narrative 
observation; 2) a teacher interview; and 3) pupil interviews with randomly 
selected students. Results from all three of these sources are documented 
in Appendix A. 

What follows in this section is a brief description of the evaluation 
process as it occured for 1979-80, In addition, we briefly document how we 
used our experiences and data from die 19 79-80 evaluation in order to refine 
and develop the model for L980-81. 

Strut tured Narrative Observation 

Kor the structured narrative observation, each observer focused on 
the same categories and wrote comments related to those categories during an 
on-site visit. The categories were: 

L. Focus of the activities 

2. Pupil thinking and response levels 

3. Teacher questioning 

4. Classroom conditions 
r 5. Overall evaluation 

After piloting the structured narrative lormat for the J979-80 
observations, we used the information gained from those exper iei .s and refined 
the observation categories so that tiiey wouJd yield more quantiJ iable daia. 
The resulting observation foim is the Gifted Class Activities Observation 
Scale (COAOS). This form conbines Lhe advantages of a narrative observat ion 
format with the easy interpretation of a more quantil iable instrument. lhe 
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GCAOS i.s further explained on page 32 

Teacher Intc rvlew 
The teacher interview used in tlu- 1979-80 model consisted of 4 
question:* : 

1. What do you like best about the program? 

2. What would you change about the program? 

3. Is teaching in this program professionally rewarding? 

4. How do you feel about the curriculum for the gifted class 
you taught? 

The observers recorded comments from teachers which are included in 
Appendix A. After using the teacher interview format, it was clear that the 
teachers were a rich source of data regarding the program and could provi 
additional useful information* Therefore, in the 1980-81 model the teacher 
ihterview was retained, and the Teacher Questionnaire was added which will 
allow teachers to assess classroom conditions, pupil learning, and attitude 
toward the program. Additionally, the Lesson Evaluation form has been added 
to the L980-81 evaluation model. This form encourages teachers to focus on 
each of the individual program goals for every session; teachers evaluate 
lesson effectiveness according to how well the objectives were met by the 
students. ' Copies of the Teacher Questionnaire and the Lesson Evaluation can 
be found in Appendix B. 

Student Interview 
Students were interviewed at random in the Spring of 1980. The 
student interview questions were: 

1. Do you like Che progcam? 

2. What do you like about the progni.ii? 

3. What would you change about the program? 
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4, How is this progran different from your regular class? 

5. How do you like the teachers in this program? 

Sample comments from the student interviews are lound in Appendix A, 
After using the student interview format for one year, we decided that 
not only should it be retained, but student input should be expanded. Therefore, 
the^Student Questionnaire (Appendix B) was developed to be administered to all 
students. This questionnaire samples student attitude tuward the program, 
teachers, and his or her own learning. 
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The 1980-81 Model for livaluating Gifted Pro grams 

in the previous section we have briefly described the 19/9-80 evaluation 
model with its 3 components; 

a) structured narrative observation form 

b) teacher interviews 

c) student interviews 

We have also noted how each of these 3 components, used in pilot form in 
SprLng 1980, was further refined for use in 1980-81. Wc then combined our 
practical experience with evaluation of gifted programs with a review of the 
available theoretical literature related to evaluation of gifted programs. 
The result of this combination of theory and practice is what we believe to be 
a workable model which addresses the evaluation needs of Lhe gifted program 
for Carroll County. 

The model matches evaluative input with program data sources to 
provide outcomes which may then be evaluated* The input topics selected were 
the gifted student identification process, cognitive student growth as a 
result of the gifted program, levels of thinking, classroom conditions and 
attitude toward the program. 

The data sources were students, program teachers, and program parents. 
Each cell of the model indicates the outcomes or processes used to evaluate 
the input data with regard to students, program teachers or program parents. 
Since parents were not involxed in the instructional aspect of this program, 
only their assessment of pupil cognitive growth and attitude was determined 
to be useful for a program evaluation. 
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A Model for Gifted Program Evaluation 



Input \ 


Identification 
Pi ocess 


Cognitive Student 
Growth 


Levels of ^ 
Thinking 


Classroom 
Conditions 


Attitude , 
Towa rd 
Program 


1 Program Data 
™ Sources 

Students 

Program 
Teachers 

Parents 


1. Student 
Achievement 

2. Aptitude Test 
data 

3. Teacher Obs. 
Worksheet . 


Selected (Pre-Post) 
Tests to evaluate 
program components 

n 


1. Outside 
evalua- 
tor : 

a. Use of a 
timed ob- j 
servation 
scale 

2. Pupil 
interview 


1. Outside 
evaluator: 

a. Eva! uation 
checklist 

2. Pupil 
question- 
naire 


1. question 

2. Pupil 
inter- 
view 


1. Principal and 
Supervisor 
Recoinmenda-% 

tions 

2. Teacher 
interview 


1. Lesson rating 
scale for each 
session 

2. Teacher ques- 
tionnaire 


1. Outside ■ 
evaluator 

a. Timed 
observa- 
tion 

2. Teacher 
inter- 
views 


1. outside 
evai uator : 

a. evaluation 
checklist 

2. Teacher 
question- 
naire 


1. question 

2. Teacher 
inter- 
view 




Questionnaire 
(Post) 


X 


\ / 

A 


Question- 
naire 
(Post) 



( input )X(Program Data Sources)-= Outcomes to Evaluate 
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Let's consider use of the Model in an evaluative setting. Input 
data will be considered with respect to program evaluation. \n outside 
evaluator could easily review the student identification process as shown 
in the model presented. Data for this identification, from tests and teachers 
would be readily available. Narrative information on gifted program teacher 
selection based on recommendations and interview comments is also readily 
obtainable. 

As 4 mentioned earlier, cognitive student growth is a difficult task 

v & 2 S 

in gifted program evaluation, Renzulli , Stanley , and others ote that 

Rre-Post use of norm referenced testing is usually inappropriate for highly 

able students. What is suggested in this Model is the use of locally 

developed criterion referenced Pre-Post teste to indicate mastery of those 

program components that, in fact, can be mastered. For those program aspects 

not measurable in a typical test setting, creative response instruments are 

suggested. The pretest aspect here is helpful in determining grouping, 

pace and actual content topic presentation within the program. Program 

teachers will determine student growth as it relates ro individual lesson 

success and their own rating of the program utilizing instruments found on 

pages 34 and 35 in Appendix B. The individual lesson rating scales 

should prove helpful as a tool for self-evaluation. 

Assessment of levels of thinking will be evaluated with regard to 

student response and teacher questioning. An instrument entitled Gifted Class 

Activities Observation Scale (GCAOS) was developed based on the Spring 1980 

evaluative efforts. This instrument is described and presented on pages 33 

of Appendix B of this report. The timed use of this scale should effectively 

assess the utilization of higher leveL response and quest iouing within the 
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classroom, as well as classroom conditions. 

Brief pupil and teacher interviews based 'on the interview questions 
used within the mathematics evaluation, complete the levels of thinking input 
information. 

As mentioned earlier, the GCAOS instrument will be utilized to assess - 
classroom conditions. An outside evaluator will conduct this observation as 
well as the pupil - and teacher interviews found in the levels of thinking input 
area. The pupil questionnaire found on page 39 . will be completed at the end 
of the gifted program to provide student input regarding classroom conditions. 
The teacher questionnaire mentioned with regard to cognitive student growth 
also contains % items designed to assess classroom conditions. 

The final, yet extremely important, input aspect is the attitude toward 
the program. The pupil and teacher questionnaire mentioned earlier contain 
items to assess program attitude. The parent questionnaire mentioned with 
regard to student growth contains items designed to measure parent atlitudej 
toward the gifted program. Additionally, pupil and teacher interview questions 
relate to program attitude. 

The Model for Gifted Program Evaluation presents a variety of related 
outcomes with which a narrative report reflecting back to data may be developed. 

The instruments referred to in this mo<!el rev leu are presented for 
consideration. It is felt that the model and related instruments provide for 
a viable, yet not overly burdensome, evaluative process. 
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5 Stanley, J.C. "Use of Tests to Discover Talent" Intellectual Talent: 
Research and Developmen t (ed. by D.P. Keating.) Baltimoie, Md. : The Johns 
Hopkins University Press, 1976, pp. 3-22. 
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APPENDIX A 



1980 - 1981 Evaluation 



-•Forms 
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Documented in this appendix are all forms to be j£$ed for the 1980-81 
evaluation. Below is a brief summary of the approximate timo during the 
program when each evaluation component will be implemented. 

P ra Assessment JLSSS. 

Pupil Identification N/A 

Teacher Ide itif ication N/A 

Criterion Referenced Tests N/A 

Ongoing Assessment , 

Lesson Evaluation 29 

Teacher Interviews 30 

Pupil Interviews 31 

Outside Evaluation (GCAOS) - 32 



Post Assessment 



* by 
outside 
evaluator 



* / 

Criterion Referenced Tests -* 33 

Teacher Questionnaire ^ 34 

Parent Questionnaire 36 

Pupil Questionnaire 39 
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^pjectives 

Uisted from 
curriculi guide 
by number) 


Met by 
fewer than 
50% of the 
students 


Lesson 

Met by 
at least 
50% of the 
s tudents 


Kvaluation 

Met by 
at least 
70% of the 
students 


Met by 
at least 
80% of the 
students 


Session No. 

Met by 
at least 
90% of the 
s tudents 
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TEACHER INTERVIEW QUESTIONS 



1. What do you like best about the program? 



2. What would you change about the program? 



3. Is teaching this program professionally rewarding? 



4. How do you ^eel about the curriculum for the gifted class 
you taught? 



n 
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PUP LL INTERVIEW QUESTIONS 



1. Do you like the program? 



2. What do you like about the program? 



3. What would you change about the program? 



4< How is this ^program different from your regular class? 



5, How do you like the teachers in this program? 
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The CCAOS assesses four major areas related to gifted class 
environment. The areas of classroom focus, pupil thinking-response, 
questioning, and classroom concerns are composed of various des- 
criptions which are to be evaluated by direct timed observation, 
(focus, thinking- response, and questioning), and rating (classroom 
concerns). The literal, critical and creative levels of thinking, 
response and questioning are defined as: 

literal - activities calling for thought, response or 

questions ranging from recall or rote knowledge 
\ through the Comprehension level as identified by 
Bloom's Taxonomy . 
critical - activities calling for thinking or responding 
based on use o£ appropriate methods for an 
analytical response. This corresponds to 
Bloom's Application and Analysis levels, 
creative - activities which result in responses that are 

newly generated and may represent standards for 
judgment. This corresponds to Bloom's Synthesis 
and Evaluation levels. 

The timed observation component of the GCAOS is to be tallied in a 
ten minute setting and should assess classroom focus, pupil thinking, 
responding, and teacher questioning. 
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Class 



Classroom Focus 0-10% L0-40% 40-70% 70-100% 

Lecture 

Discussion 

Activity 

(Tally) Pup LI Thinking-Response 0-10% 10-40% 40-70% 70-100% 

Literal 

Critical 

Creative 

(Tally) Teacher Questioning 0-10% 10-40% 40-70% 70-100% 

Literal 

Critical 

Creative 

(rating) Clas sroom Conditions poor fair 0.: good outstanding 

Pupil: a. On Task Behavior 

b. Interaction 

c. Involvement in Decision 

Making 

d. Enthusiasm 

Teacher: a. Tolerance for Divergent 
Responses 

b. Allowance for humor 

c. Delivery 

d. Empathy 
Classroom: a. Control 

b. Time Management 

c. Utilization of 

Materials 



Evaluator Comments: 



Evaluator 
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Teacher Questionnaire 

Circle tlie response which you feel is most appropriate. 
1. Circle che number which in your opinion best describes classroom 
conditions . 



a) teaching/learning 
atmosphere 

b) teaching/ learning 
atmosphere 

c) pupil input 



d) pupil/pupil 
interaction 

e) degree of 
teacher control 

f) amount of pupil 
on-task behavior 



closed 
1 

tense 
i 

limi ted 
1 

limited 
1 

limited 
1 

limited 
1 



3 
3 
3 



4 
4 
4 
4 
4 



open 
5 

i 

relaxed 
5 

extensive 
5 

extensive 
5 

extensive 
5 

extensive 
5 



2. Circle the number which in your opinion best describes pupil 
in this class. 



learning 



a) degree of 
motivation 

b) ability of pupils 

to work independently 

c) ability of pupils 

to work cooperatively 

d) extent to which 
students accomplished 
objectives 



limited 
1 



poor 
1 



poor 
1 

limited 
1 



3 
3 



extensive 
5 

excellent 
5 

excellent 
5 

extensive 
5 
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3, Circle the number which best describes your attitude toward the 
pro gram » 



a) worth of the pro- limited 
gram for students 1 

b) worth of the pro- limited 
gram for your pro- 1 
fessional growth 

c) your confidence in limited 
working with gifted 1 
students 

disliked 

d) your attitude toward it 
teaching in the pro- * 1 
gram 

e) your preparation for poor 
teaching in the pro- 1 
gram 

f) overall evaluation poor 
of program 1 



3 
3 



extensive 
5 

extensive 
5 



extensive 
5 

thoroughly 
enjoyed it 
5 



excellent 
5 



excellent 
5 
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Gifted Program 
Farent Attitude Quebtionnaire 

rections - Circle the most appropriate response. 

1. My child seemed to learn much by his/her involvement, in the gifted program. 

a. strongly agree 

b. agree 

c. .neutral 

d. disagree 

e. strongly disagree 

2. My child usually shared ideas learned in the gifted program with me/us. 

a. strongly agree 

b. agree 

c. neutral 

d. disagree 

e. strongly disagree 

3. I would like my child to continue with this program, or a similar one, next 
year. 

a. strongly agree 

b. agree 

c. neutral 

d. disagree 

e. strongly disagree 

4. I felt the gifted program was excellent, 
r. strongly agrse 

b. agree 

c. neutral 
(I. disagree 

e. strongly disagree . 
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5. The teachers for my child's gifted program were well liked. 

a. strongly agree 

b. agree 

c. neutral 

d. disagree 

e. strongly disagree 

6. 1 felt the gifted program would have been better if held after school. 

a. strongly agree 

b. agree 

c. neutral 

d. disagree 

e. strongly disagree 

7. I wouLd not want my child in a gifted program in language arts or mathematics 
again. 

a. strongly agree 

b. agree 

c. neutral 

d. disagree 

e. strongly disagree 

8. Gifted programs should be expanded i.i Carroll County. 

a. strongly agree 

b. agree 

c. neutral 

d. disagree 

e. strongly disagree 
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9. The ideas learned in the gifted program were not useful to my child, 

a. strongly agree 

b* agree 

c. neutral 

d. disagree 

e. strongly disagree 

10. My child's attitude toward language arts /mathematics has improved as a 
result of the gifted program. 

a. strongly agree* 

b, agree 
c* neutral 
d, disagree 

e* strongly disagree 

IK Have you been provided with enough information about the Gifted Program? 

Yes No 

12, Have you had an opportunity to discuss your child's progress with the 
gifted class teachers? 

Yes No 



13. Did you feel the gifted program was challenging for your child? 

Yes No 
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STUDENT QUKSTI0NNA1KK 



Directions : Circie the number that tells bust how you feel. For instance, 
if the question asks "how much did you like the program?" and * 
your choices were: 



Hated It 



Really enjoyed 
it 



you might circle //l if you really hated the program, //5 is you 
really enjoyed it or //3 if you felt somewhere in between* 



1. Would you recommend this class to a friend? 

Absolutely % 
Not 

1 2-3 4 

2. How did you spend most of your time working in class? 

Alone 

12 3 4 

3. How did you feel about this class? 

Bor ing 

12 3 4 

Useless 



Absolutely 
Yes 



With several 
others 



Exciting 
5 

Worthwhile 



4. What kind of person was the teacher? 

Unfriendly 

"l 2 

Not Helpful 

1 2 

Not Inter- 
ested in me 



Friendly 
5 

Very Helpful 
5 

Very interested 
in me 
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5, How did you feel about your own learning in this class? 



Didn't 

]earn 

much 



Learned 
a 
lot 



Didn't 

like 

it 



Liked 

It 
A Lot 



How much cf the time during class were you doing what you were supposed 
to be doing? 



Very 
Li t tie 



Almost 
All 
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